Accelerating mini-batch SARAH by step size rules

StochAstic Recursive grAdient algoritHm (SARAH), originally proposed for convex optimization and also proven to be effective general nonconvex optimization, has received great attention because of its simple recursive framework updating stochastic gradient estimates. The performance SARAH significantly depends on the choice step size sequence. However, variants often manually select a best-tuned size, which is time consuming in practice. Motivated by this gap, we propose variant Barzilai-Borwein (BB) method, referred as Random (RBB) determine mini-batch setting, leading new method: MB-SARAH-RBB. We prove that MB-SARAH-RBB converges linearly expectation strongly objective functions. Moreover, analyze complexity show it better than original method. To further confirm efficacy RBB MB-SARAH+-RBB incorporating into MB-SARAH + Numerical experiments standard data sets indicate our methods outperform or match state-of-the-art algorithms.